Recent electronic camera designs typically use planar CCD and CMOS type sensors. In order to provide a succession of color image frames wherein each frame has full color content using these sensor types, one of two approaches is used. In one method, three separate sensor arrays are provided, with either a red, a green, or a blue filter in front of each sensor array. Alternatively, a prism could be used to split incoming light into three colors, with each color provided to a separate, unfiltered sensor array. This first method provides a tristimulus red, green, and blue (RGB) value for each pixel.
A second method uses a single sensor array and places a color filter array (CFA) over the sensor array such that a red, a green, or a blue filter lies over each sensor of the sensor array. With this second method, since the full tristimulus RGB value is not obtained from each sensor of the sensor array, interpolation is used to calculate missing values, based on the matrix of values obtained. The second method has been used for lower resolution sensors, especially for consumer still cameras, so that images obtained on these cameras can be output easily to printing and display devices in standard TIFF or BMP (bitmap) formats. Although the second method may introduce some unwanted effects in an image under certain conditions, this method has the advantage of eliminating color misregistration errors associated with misaligmnent of multiple sensors and is often used for generating NTSC television signals.
Yet another method is available when using a single sensor, but does not provide simultaneous RGB color content. This third method employs a field sequential camera with a single sensor and a rotating color filter wheel with red, green, and blue filters.
Among patents that describe interpolation techniques used with the second method given above, U.S. Pat. No. 3,971,065 (Bayer) discloses a color imaging array wherein a mosaic of selectively transmissive filters is superimposed in registration with a solid imaging array. In a preferred embodiment, each row contains alternating filters for luminance and a first chrominance and alternating rows contain luminance filters alternating with a second chrominance filter. The advantage of this approach, wherein there are twice as many green pixels as red or blue pixels, is that a higher resolution is obtainable in green, to which the eye is most sensitive. Green sensitivity is also most closely related to the luminance channel value for a color image. As is well known in the imaging arts, the human eye is most sensitive to luminance and much less sensitive to chroma information. Thus, luminance data is important in NTSC color transmission used for color TV, for example. Other examples of interpolation techniques used with digital color cameras that employ CFAs are disclosed in U.S. Pat. No. 5,990,950 (Addison) and U.S. Pat. No. 6,181,376 (Rashkovskiy et al.)
For video camera applications, U.S. Pat. No. 5,251,019 (Moorman et al.) discloses a solid state color image sensor used with a CFA. The color filter array in U.S. Pat. No. 5,251,019 covers an array of image sensor elements wherein 75% of the image sensing elements are luminance sensing, for example, green elements, and the rest are chrominance sensing, for example, red and blue elements.
With the goal of improving image quality, there have been a number of solutions proposed for improving the RGB sensitivity of digital color cameras. As one example, instead of using a color filter array (CFA) of red, green, and blue (RGB) filters, using a filter array of cyan, magenta, and yellow (CMY) filters has been proposed, as noted in U.S. Pat. Nos. 5,631,703 and 6,330,029 (Hamilton et al.) Accurate RGB values can then be derived algebraically from the CMY values. It is further noted in the latter patent as an advantage of such a filter technique that twice the amount of energy falls on each pixel as compared to the RGB color filter array (CFA). For example, cyan transmits both blue and green light, so that the detector cell would see twice as much light as it would with either a blue or a green filter. This advantage provides a better signal to noise ratio for a given cell size and integration time. In a preferred embodiment, because luminance information is derived from the green channel, an additional green filter is also added to the cyan, magenta, and yellow filters to provide a better interpolation of RGB values for each pixel.
In another attempt to improve camera performance with various light sources while minimizing interpolation artifacts, U.S. Pat. No. 5,889,554 (Mutze) discloses the use of five color filters and preferred patterns for arranging them. The preferred colors are B (455 nm), G′ (494 nm), G (545 nm), G′ (570 nm), and R (595 nm). The extra colors aid in improving the interpolation of RGB values for each pixel; no additional color data is provided.
With the goal of improving CCD sensor performance through device manufacturing techniques, U.S. Pat. No. 6,001,668 (Anagnostopoulos) describes the use of transparent ITO electrodes in sensor fabrication. With a similar goal, U.S. Pat. Nos. 5,677,202 and 5,719,074 (Hawkins et al.) disclose improved methods of manufacturing CFAs onto CCDs.
The above cited patents show attempts at improving color quality of digital color images by making incremental improvements to the RGB data as acquired and processed by a digital camera. Referring to FIG. 10, there is shown a familiar graphical representation of the human-visible color gamut, shown as a horseshoe-shaped periphery 100. Within periphery 100 are represented two smaller color gamuts: a motion picture film color gamut 102 and an NTSC TV color gamut 104. It is instructive to note that the color gamut is essentially defined by a triangle, where each vertex corresponds to a substantially pure color source, ideally a primary color, that serves as a component color for other colors within the gamut. The area of the triangle thus represented corresponds to the size of the color gamut. To expand color gamut requires moving one or more vertices closer to periphery 100.
Conventional color models, such as the CIE LUV model that follows the color space conventions defined in 1931 by the Commission Internationale de l'Eclairage (International Commission on Illumination), represent each individual color as a point in a 3-dimensional color space, typically using three independent characteristics such as hue, saturation, and brightness, that can be represented in a three-dimensional coordinate space. Color data, such as conventional image data for a pixel displayed on a color CRT, is typically expressed with three-color components (for example R, G, B), that is, in tristimulus form. Conventional color projection film provides images using three photosensitized emulsion layers, sensitive to red, blue, and green illumination. In fact, the human eye itself has three-color sensors, R, G, B. Because of these conventional practices and image representation formats, developers of cameras, films, printing apparatus and display systems have, understandably, adhered to a three-color model.
There have been some attempts to expand from the conventional three-color model in order to represent color in a more accurate, more pleasing manner. Notably, few of these attempts are directed to expanding the color gamut. For example, the printing industry has used a number of strategies for broadening the relatively narrow gamut of pigments used in process-color printing. Because conventional color printing uses light reflected from essentially white paper, the color representation methods for print employ a subtractive color system. Conventionally, the process colors cyan (blue+green), magenta (red+blue) and yellow (red+green) are used for representing a broad range of colors. However, due to the lack of spectral purity of the pigment, combinations of cyan, magenta and yellow are unable to yield black, but instead provide a dark brown hue. To improve the appearance of shadow areas, black is added as a fourth pigment. As is well known in the printing arts, further refined techniques, such as undercolor removal could then be used to take advantage of less expensive black pigments in full-color synthesis. Hence, today's conventional color printing uses the four color Cyan, Magenta, Yellow, and blacK (CMYK) method described above. However, even with the addition of black, the range of colors that can be represented by printing pigments is limited.
Other examples showing where additional color components have been added to improve color appearance are from digital projection apparatus. U.S. Pat. No. 6,256,073 (Pettit) discloses a projection apparatus using a filter wheel arrangement that provides four colors in order to maintain brightness and white point purity. However, the fourth color added in this configuration is not spectrally pure, but is white in order to add brightness to the display and to minimize any objectionable color tint. It must be noted that white is an “intra-gamut” color addition; in terms of color theory, adding white actually reduces the color gamut by desaturating the color. Similarly, U.S. Pat. No. 6,220,710 (Raj et al.) discloses the addition of a white light channel to standard R, G, B light channels in a projection apparatus. As was just noted, the addition of white light may provide added luminosity, but constricts the color gamut. U.S. Pat. No. 6,191,826 (Murakami et al.) discloses a projector apparatus that uses four colors derived from a single white light source, where the addition of a fourth color, orange, compensates for unwanted effects of spectral distribution that affect the primary green color path. Again, the approach disclosed in the Murakami patent does not expand color gamut and may actually reduce the gamut.
Unlike the earlier patents listed above for projection apparatus, Patent Application WO 01/95544 A2 (Ben-David et al.) discloses a display device and method for color gamut expansion using four or more primary colors. However, while the methods and apparatus disclosed in application WO 01/95544 provide improved color gamut for projected images, the image data that is originally input to the projection device is tristimulus RGB data, not four-color data.
Thus, it can be seen that it would be advantageous to provide a camera which could provide a signal having a fourth color that would result in an improved color gamut. Such a signal could be input to a projector mechanism or printing device that could take advantage of this extended gamut and provide a more pleasing image.